The process of identifying talent in soccer involves multiple factors, with technical proficiency often considered a key indicator of performance potential (for review, see (Sarmento et al., 2018). Research highlights the importance of technical skills, such as dribbling, passing, and shooting accuracy, as critical metrics for coaches and scouts to assess a player’s abilities (Bergkamp et al., 2022; Huijgen et al., 2009; Waldron and Worsfold, 2010; Williams et al., 2020). The application of these technical skills in soccer requires players to adapt to the dynamic and ever-changing demands of the game environment (Araújo et al., 2019). Furthermore, the adaptive actions of one player contribute to the formation of team synergies, enabling cohesive and effective collaboration (Araújo et al., 2023).

As a result, players develop attunement to information relevant to specific goal-directed behaviors, which enhances the team’s ability to respond dynamically and efficiently to the immediate challenges of the performance environment (Button et al., 2021). This dynamic interplay highlights the importance of perceptual-cognitive elements in soccer, including aspects such as the speed and accuracy of stimulus localization, working memory, motor response control, anticipation, and decision-making (Bennett et al., 2019; Glavaš et al., 2023; Woods et al., 2020; Zhao et al., 2022).

In this context, exploring new methods for assessing soccer players' performance that integrate cognitive skills in soccer-related scenarios becomes crucial. Current testing procedures often evaluate cognitive skills in laboratory settings and controlled scenarios (Feria-Madueño et al., 2024; Glavaš et al., 2023; Zwierko et al., 2019). However, applying laboratory conditions to ensure repeatability and reproducibility decreases a test’s ecological validity, i.e., how well a test predicts performances in real game scenarios (Kihlstrom, 2021), and typically does not include actions specific to soccer players. On the other hand, traditional field-based tests in soccer training primarily focus on assessing motor abilities, physiological parameters, tactical understanding, and technical skills (Clemente et al., 2022; Dambroz et al., 2022), making it challenging to capture significant cognitive skills. Similarly, popular game-based tests, such as the Game Performance Assessment Instrument (GPAI) (Oslin et al., 1998), the Game Performance Evaluation Tool (GPET) (Memmert and Harvey, 2008), and the System of Tactical Assessment in Soccer (FUT-SAT) (Costa et al., 2011), provide valuable insights into players' game performance and tactical competencies. However, their effectiveness can be limited by subjective evaluation methods, observer reliability issues, and difficulties in fully replicating the cognitive demands of competitive matches (García López and Gutiérrez Diaz del Campo, 2018; Memmert and Harvey, 2008). Therefore, it is essential to explore complementary methods that can support and enhance the current approaches to player assessment, ensuring a more comprehensive evaluation of soccer performance.

As new technologies advance, there is a growing demand for supplementary training and testing methods aimed at enhancing sports performance, particularly among young soccer players developing their motor-cognitive skills. Since success in soccer relies on a combination of key competencies, innovative approaches to sports diagnostics and training that integrate motor and cognitive skills hold promising potential (Friebe et al., 2024a; Friebe et al., 2024b; Höner et al., 2023; Musculus et al., 2022). For example, recent proposals for assessing motor-cognitive agility performance in elite youth soccer players have focused on the use of Skillcourt technology (Friebe et al., 2024b), which integrates multiple object tracking into reactive agility testing. However, in this case, while typical movement patterns involving changes of direction in response to unexpected stimuli were included, the object tracking component involved non-specific stimuli. In this context, the development of new diagnostic tools that integrate objective measurements of motor, technical, and cognitive aspects of soccer performance appears highly justified. In that sense, our research focuses on developing a novel test battery aimed at improving the prediction of on-field performance. This approach integrates complex technical and motor elements with cognitive tasks, placing a strong emphasis on assessing performance in specific sports contexts.

The Skills.Lab Arena is an advanced simulator designed to integrate motor and cognitive tasks across various technical and tactical skills, offering a structured platform for assessing soccer performance in controlled yet specific conditions. Covering an area of 320 m², the facility is equipped with ball launchers, cameras for tracking ball trajectory and player movements, and six interactive screens displaying dynamic visual stimuli, such as moving players and targets. These features enable precise evaluation of performance metrics, including shot and pass speed and accuracy. Despite its implementation in top soccer academies, there is currently a lack of scientific research exploring its potential in structured studies. This presents an opportunity to systematically evaluate the reliability and usefulness of Skills.Lab Arena in assessing and training soccer-specific motor-cognitive skills under conditions closely replicating on-field scenarios. Given that the proposed test battery combines complex motor and technical and tactical components with complex cognitive tasks, this integration may induce high intertrial variability. Therefore, the first step in this evaluation focuses on assessing reliability, as it is critical for determining whether the test is practically applicable and for ensuring consistent results in subsequent assessments. Establishing reliability is a prerequisite for meaningful validation, as a measurement process must demonstrate sufficient stability before its validity can be properly assessed (Friedman et al., 2022).

In that sense, this study aims to evaluate the psychometric properties of ten test tasks designed specifically for U-14 soccer players, utilizing the Skills.Lab Arena technology. By ensuring the reliability of the test battery, we aim to provide a robust tool that enhances talent identification, optimizes training interventions, and improves performance assessment in soccer academies. Standard reliability measures, such as the intraclass correlation coefficient (ICC), Bland-Altman analysis for agreement assessment, and coefficient of variation (CV), were employed. The study aims to demonstrate the usefulness of the test battery as a tool sensitive enough to detect performance changes that are practically significant and not merely due to random fluctuations caused by measurement error (Atkinson and Nevill, 1998). To achieve this, the typical error (TE) and smallest worthwhile change (SWC), key parameters for evaluating test usefulness, were calculated.

Thirty-one elite youth male soccer players aged 13 to 14 years (mean age = 13.5 ± 0.5 years, body mass = 52 ± 7 kg, height = 1.60 ± 0.08 m) participated in the study, providing sufficient statistical power for the analysis. An a priori power analysis was conducted using G*Power (version 3.1.9), based on a within-subjects analysis of variance (ANOVA) as the main statistical test, with an α-level of 0.05, power of 0.9, and effect size (f) of 0.33, based on the average effect size obtained in the study by Friebe et al. (2023) on the reliability of motor-cognitive testing tools, including agility and reaction time tasks. The analysis indicated that a minimum sample size of 21 participants was necessary to achieve sufficient statistical power, a criterion that was met in our study. All participants had at least three years of competitive experience and trained regularly at a local soccer club, practicing four times a week and competing in a championship match each weekend. This consistent training and match play provided a strong foundation for evaluating their performance in various soccer-specific tasks. Prior to the study, informed consent was obtained from all players, and their legal guardians gave written consent to participate in the study. The study protocol adhered to the tenets of the Declaration of Helsinki and was approved by the University Institutional Review Board under the reference number 92/24.

Measurement of football players' motor and cognitive skills was conducted on a grass pitch at Skills.Lab Arena (Anton Paar SportsTec GmbH, Austria). The simulator was equipped with 4 ball machines (Machines 1 to 4) capable of reaching speeds up to 130 km/h, and 6 cameras for analyzing the trajectory of the balls, with a resolution of 2448 × 2048 pixels and an image capture speed of 40 frames per second. The ball machines featured adjustable speed and trajectory settings, allowing for a wide range of training scenarios, from low-speed passes to high crosses. Each machine was designed for precision and reliability, integrated seamlessly with the Skills.Lab system for accurate monitoring and analysis. Around the pitch, there were 6 screens (Screens 1 to 6) with a total area of 60 m², made of block tarpaulin measuring 9.85 × 4.70 m. Each location where the ball hit one of the screens was recorded by the system with an average deviation of 3.6 cm. The Skills.Lab Arena was managed by servers running on a 64-bit Linux Mint 21.2 operating system, ensuring reliability and performance. Examples of the test setups in the Skills.Lab Arena are presented in Figure 1.

The evaluation session in the Skills.Lab Arena included 10 tests designed to assess various aspects of U-14 football skills, including: passing accuracy, shooting accuracy, ball control, cognitive abilities, and motor skills. Each task assessed the percentage accuracy of passes and/or shots (Tasks 1 to 8), as well as the time taken to complete each action.

The player was positioned facing Machine 1, which fired 10 balls in sequence, at a speed of 40 km/h. The task required the player to make quick and accurate decisions in response to simple stimuli by receiving each ball and then executing a precise pass to a designated target. These targets were displayed alternately on the left and right screens, demanding rapid reactions to changing conditions and continuous adjustments in positioning and technique. The test assessed the player's technical skills in ball control and passing as well as their ability to make fast decisions under pressure (Video TEST1).

In this exercise, the player had to demonstrate not only technical precision in receiving the ball, but also the ability to make quick decisions and anticipate the movement of the target. Positioned in front of Screen 2, the player received 10 passes alternately from Machines 1 and 2, which fired the ball at a speed of 45 km/h. On the screen, a player simulating a running teammate appeared across the full width of the screen, requiring the player to deliver a precise and timely pass. This test measured the player’s ability to evaluate the situation dynamically, adjust body positioning, and choose the most effective foot for the pass in response to a complex, changing environment (Video TEST2).

This task aimed to assess both the power and precision of the shot, as well as the ability to adjust the shot based on the side from which the ball was received. The player received eight balls alternately from Machines 1 and 2, fired at a speed of 40 km/h. Screen 2 displayed a large goal with a goalkeeper, requiring the player to quickly decide which foot to use for the shot. The test evaluated the player's ability to deliver powerful and accurate shots while quickly adapting to changes in the side from which the ball was passed on the field (Video TEST3).

The player needed to demonstrate not only technical precision but also cognitive aiming—the ability to target small, randomly appearing goals on the screen. Positioned next to the scoreboard, the player received 12 balls alternately from Machines 1 and 4 at a speed of 45 km/h. On Screen 3, a goal appeared, with small targets randomly displayed. After receiving the ball, the player needed to quickly assess the situation and decide on the appropriate foot to use based on the origin of the pass. This task evaluated quick decision-making, precise aiming, and adaptability to changing conditions (Video TEST4).

In this task, the player had to combine the ability to quickly process information with the technique of receiving and delivering precise passes under time pressure to a player moving from the right or left side on the screen. Positioned next to Screen 5, the player received 10 passes alternately from Machines 1 and 3, fired at a speed of 60 km/h.

After each pass, the player’s task was to quickly receive the ball, dribble it to the circle displayed on the field, and then pass to a running player displayed on Screens 2 and 3, requiring appropriate timing and pass accuracy. The screens also showed static opponents (Video TEST5).

This test required mastering the technique of receiving and passing the ball alongside the ability to execute complex reactions, such as completing a 180-degree turn in response to changing screen conditions. Positioned in the center of the Arena, the player received 20 passes alternately from Machines 1 and 4, fired at a speed of 50 km/h. The screens displayed static players to whom the ball needed to be passed accurately after completing the turn. The player needed to process information rapidly to adjust their position and passing technique based on the incoming pass direction and target location (Video TEST6).

This task combined anticipation of screen-displayed player movements with quick decisions on passing direction after a turn. Positioned in the center of the Arena, the player received 20 passes alternately from Machines 1 and 4, fired at a speed of 55 km/h. After receiving the ball, the player performed a 180-degree turn and, while running, passed the ball to a moving target displayed on the screen. The presence of static defenders required continuous adaptation of technique and decision-making to maintain possession (Video TEST7).

This task emphasized the player’s cognitive abilities, such as rapid perception, working memory, and situational awareness, to quickly analyze the situation on the field and identify the numerical advantage of teammates. Positioned in the center of the Arena, the player received 10 passes alternately from Machines 1 and 4 at a speed of 45 km/h. The screens displayed running players and opponents, requiring the player to identify the screen with a numerical advantage (e.g., 3:2, 2:1, 1:0). The player then executed a precise pass while running, integrating perceptual, cognitive, and motor skill. This test assessed the ability to analyze field situations quickly, recall tactical patterns, and make effective decisions under pressure (Video TEST8).

The player demonstrated agility and speed while dribbling the ball in a dynamically changing environment. Positioned next to Machine 4, the player received a pass from this machine at a speed of 25 km/h and had to dribble the ball as quickly as possible within a rectangle measuring 3 × 5 meters. This task tested agility and technique adjustments to the changing conditions (Video TEST9).

This task was similar to Task 9 but did not involve a ball. The player started in an illuminated laser circle, with the timer starting as they exited the circle. The run ended when the player returned to the illuminated circle, completing two laps (Video TEST10). Table 1 presents a summary of the description of the 10 tests.

To assess the reliability and usefulness of the Skills.Lab Arena, we employed a test-retest study design. Following the guidelines from previous reliability studies in sports (Hopkins et al., 2001; Schatz and Ferris, 2013). we set the measurement intervals at 7 days and 1 month. On each of the three testing days, participants completed the entire set of test tasks, numbered from 1 to 10, in the order recommended by the manufacturer. Prior to the tests, participants underwent a 15-minute warm-up, which included jogging, joint mobility exercises, dynamic stretching, and short sprints. Before starting the tests, an investigator provided each participant with standardized instructions. Subsequently, each participant completed an adaptation trial to familiarize themselves with the adaptation task procedure in the Skills.Lab Arena, which included 2 exercises with a ball. The adaptation task lasted 3 minutes. The entire measurement procedure for one person took approximately 35 minutes.

After an initial plausibility check, the assumptions for parametric testing required for ANOVA were verified. The Shapiro–Wilk test was used to assess normality, and Levene’s test was applied to evaluate the homogeneity of variances. Descriptive statistics were expressed as mean ± standard deviation (SD) and 95% confidence intervals (CI).

To evaluate reliability and usefulness, both absolute and relative reliability were assessed. Absolute reliability was expressed as the typical error (TE), calculated using Hopkins's spreadsheet, and reported as a coefficient of variation (CV%) (Hopkins, 2000). A CV of less than 10% was considered the threshold for reliability (Simperingham et al., 2016). The smallest worthwhile change (SWC), representing a small effect, was calculated as 0.2 × between-subject SD. Tests were considered sensitive and useful if SWC exceeded TE (Hopkins, 2004).

Relative reliability was assessed using intra-class correlation coefficients (ICC, model 3,1). An ICC ≥ 0.7 was deemed acceptable for indicating good agreement and the ability of the test to differentiate between individuals (Baumgartner and Chung, 2001). Additionally, Bland-Altman analysis was performed to assess agreement by evaluating mean difference and limits of agreement (LoA) (Bland and Altman, 1986). In this study, the first two measurements (1 and 2) were averaged to represent a combined baseline estimate, which was then compared to the third measurement (3).

Potential learning effects were analyzed using repeated-measures ANOVA, with "days" (day 1, day 2, day 3) as the factor. Post-hoc comparisons with Bonferroni adjustments were conducted to determine differences between days. Effect sizes were classified according to partial eta squared (ηp²) values: small (0.01), medium (0.06), and large (0.14) (Cohen, 1988). The level of statistical significance was set at p < 0.05. All analyses were conducted using the JASP statistical package (version 16.1, jasp-stats.org).

Intersession reliability results and ANOVA analyses evaluating learning effects across test days are presented in Table 2 and Table 3, respectively. All tests exhibited good intersession reliability, as reflected by ICC values across the three testing days, with time parameters showing ICCs ranging from 0.75 to 0.89, and accuracy parameters ranging from 0.71 to 0.91. Bland-Altman analysis showed mean differences ranging from -0.06 to 0.82 seconds for time parameters and from -0.32 to -0.07 points for accuracy parameters across all tests, with acceptable 95% limits of agreement. All Bland-Altman plots for each test are included in the supplementary materials (Figures S1-S17), providing a clear visualization of measurement agreement. Within-individual variation, expressed as CV, ranged from 1.78% to 4.5% for time parameters, and from 8.08% to 19.87% for accuracy. The analysis showed that the smallest worthwhile change (SWC) exceeded the typical error (TE) for all the tests examined.

ANOVA indicated significant differences in performance across the three testing days for the agility test with a ball (T9) (F_2,90 = 3.103, p = 0.050, ηp² = 0.065), with post-hoc analyses revealing notable performance improvements between the first and third testing days (p = 0.044, d = 0.6). No significant intersession variations were observed in the results of the remaining tests.

This study explored the intersession reliability and usefulness across a broad range of motor-cognitive and soccer technical and tactical skills using the Skills.Lab Arena, specifically designed for evaluating young U-14 soccer players. The relative reliability, measured by intra-class correlation coefficients (ICC), for all soccer skill measures related to ball control, passing, and shooting (T1-T9), ranged from moderate to high (ICC = 0.75-0.91). These values are significantly higher than those reported in earlier tests that focused on passing and shooting accuracy (ICC = 0.26-0.38) (Ali et al., 2007; Russell et al., 2010) and are comparable to findings from Radman et al. (2016) (ICC = 0.70-0.88). However, they are slightly lower than those observed in stationary shooting tests (ICC = 0.87-0.95) (Berjan Bacvarevic et al., 2012; Markovic et al., 2006).

The Bland-Altman analysis demonstrated a high level of agreement between test sessions, as indicated by the minimal mean differences and narrow 95% limits of agreement (Bland and Altman, 1986). The small range of mean differences for time (-0.06 to 0.82 seconds) and accuracy (-0.32 to -0.07 points) suggests negligible systematic bias. Additionally, the acceptable LoA indicates that variability between sessions was within a practically meaningful range, reinforcing the reliability of the measurements across the three test days.

Overall, the Skills.Lab Arena tests demonstrated good reliability, particularly in time-related parameters, with the coefficient of variation (CV) consistently remaining below 5%. This range of 1% to 5% for the CV is typical for most performance and sports-related tests, as these values are considered highly reliable and acceptable within the context of sports research (Hopkins, 2000). However, the analysis of accuracy parameters in the Skills.Lab Arena soccer tests related to passing and shooting accuracy showed that in most cases, the coefficient of variation (CV) exceeded 5% (Table 1). In performance tests related to soccer, particularly those involving technical skills such as passing and shooting accuracy, it is not uncommon for the CV to exceed 10% (Clemente et al., 2022). This is especially true for more complex tests that combine technical actions like passing and shooting, where higher variability is often observed. For instance, some studies have reported CV values exceeding 30% in certain soccer performance tests for young players, particularly in scenarios where technical skills are combined with physical actions (Rubajczyk and Rokita, 2015). Moreover, it is believed that soccer skills such as passing, dribbling, and shooting typically exhibit higher within-individual variation compared to measures of motor abilities such as speed, agility, and endurance (Ali, 2011).

The coefficient of variation (CV) in soccer-specific skill tests varies with the players' skill levels. Ali et al. (2007) found that professional players performed more consistently on both the Loughborough Soccer Passing Test (LSPT) and the Loughborough Soccer Shooting Test (LSST) compared to non-professional players. For the LSPT, professionals exhibited a CV of 4.7% for time taken and 11.2% for performance time, while non-elite players showed higher variability with CVs of 8.0% and 16.0%, respectively. In the LSST, elite players also demonstrated lower CVs across all metrics, indicating greater consistency: 3.5% vs. 5.1% for time taken, 8.4% vs. 10.7% for shot speed, and 49.4% vs. 57.8% for points per shot. This underscores the influence of skill level on test reliability. Similarly, Radman et al. (2016) reported a CV of 15.4% for shooting accuracy, further illustrating the trend observed in our study.

The analysis of the usefulness ensures that the tests in the Skills.Lab Arena are not only reliable but also sufficiently sensitive to detect even the smallest improvements crucial for player development. The results suggest that all tests demonstrate reliability for practical use, as the smallest worthwhile change exceeded the typical error, indicating that the observed changes in performance are meaningful and not due to measurement variability (Hopkins, 2004). Importantly, this enables practitioners working with adolescent soccer players to determine whether training interventions aimed at improving specific fitness parameters are meaningful and not merely a product of measurement error (Čović et al., 2016; McBurnie et al., 2019). In technologically advanced environments like the Skills.Lab Arena, the combination of precise measurement tools and advanced data analysis enables the accurate detection of both technical skills and cognitive abilities under game-like conditions. As a result, these tests are exceptionally useful not only for talent identification, but also for the continuous development of players (Koopmann et al., 2020).

In our study, we observed minimal variability in test results between sessions, with a slight overall trend toward improved performance in subsequent measurements. However, a learning effect was confirmed in only one instance, specifically in the agility test with the ball (T9), where significant improvements were noted between the second and third test days (Table 2). This indicates that participants demonstrated notable performance increases from the second to the third test day during this particular test. At this stage of the research, it is difficult to determine whether the observed improvement in the dribbling test is solely due to a learning effect or if it also reflects an enhancement in motor skills related to agility maneuvers over the course of a month (Altmann et al., 2022). Additionally, in Test 5, the variability in results approached statistical significance. Overall, the lack of significant learning effects for most parameters in the Skills.Lab Arena tests reinforces their effectiveness in accurately measuring soccer skills in young players. The adaptive procedures applied before the test sessions were well-designed and effectively minimized learning biases.

Despite the noteworthy findings summarized above our study had certain limitations. First, the sample size may have been insufficient to generalize the results to broader populations. In the Skills.lab Arena system, each age group is provided with a specific set of tests tailored to their soccer skills, with tasks varying in speed and technique. Research has shown that an individual’s overall skill level is strongly and positively correlated with age (Wilson et al., 2016). Additionally, these tests are designed to appropriately challenge players at their developmental stage. Second, although the test assesses both soccer skills and cognitive abilities, it treats them collectively. The test does not allow for the differentiation of cognitive parameters; however, it does reflect the typical behavior of a player during a game, which emphasizes its ecological validity. Third, the study was conducted in a laboratory setting, which, while providing control over test conditions, may have influenced the players' behavior compared to match situations on the field, where external factors such as crowd pressure, weather conditions, or interactions with other players come into play (Mann et al., 2007). Fourth, to enhance the scope of evaluation, future research should include the assessment of psychological variables, such as stress levels or motivation, which may also affect test results (Janelle and Hillman, 2003). Finally, we acknowledge the necessity for further research focused on validating the proposed test battery. While validity is typically assessed prior to reliability, the high intertrial variability inherent in the integration of motor, technical, and cognitive components required us to prioritize reliability as a foundation for subsequent validation studies. Without reliable results, any validity assessment would risk yielding inconsistent findings, undermining the test’s applicability. Future studies should focus on validating the test against soccer-specific performance metrics, such as passing accuracy, tactical decision-making, or match performance data, to ensure its ecological relevance and practical utility.

The Skills.Lab Arena tests have proven to be a reliable method for evaluating soccer skills in young players. Moreover, the test battery has demonstrated sufficient sensitivity in detecting differences in soccer ball control, passing, and shooting accuracy under time constraints in ecologically valid conditions. However, while the test provides reliable measurements, its validity requires further investigation. Therefore, additional validation steps are necessary before recommending its practical application. Nonetheless, this interactive test battery holds significant potential for future research and practical use, particularly in player selection, profiling, and assessing the impact of training interventions aimed at enhancing soccer performance.